In most compression algorithms some form of loss in the decoded picture is expected. A typical method for compression that produces good results is to introduce this loss by quantizing the signal in the transform domain instead of the pixel domain. Examples of such transforms are the Discrete Cosine Transform, DCT, the wavelet transforms and the subband analysis filters. In a transform based compression algorithm, the picture is converted into the transform domain and a quantization scheme is applied to the coefficients to reduce the amount of information. The transformation has the effect of concentrating the energy into a few coefficients and noise can be introduced into these coefficients without affecting the perceived visual quality of the reconstructed picture.
It is well known that some form of human visual perception system with different weighting on the quantization on different coefficients can improve the perceived visual quality. In coding standards such as the ISO/IEC JTC1/SC29/WG11 IS-13818-2(MPEG2), the quantization of the DCT coefficients are weighted by the quantization matrix. A default matrix is normally used however the encoder can choose to send new values of the quantization matrix to the decoder. This is done through the signaling in the bitstream header.
The prior art on sending Quantization Matrix based on the MPEG-2 video standard, is to send 64 fixed values of 8-bit each if the bit signaling for using a special Quantization Matrix is set to “1”.
The values of the matrix in the position of higher frequency band are actually not used, especially for very low bit rate coding where a large quantization step is employed, or for an input block with very plain texture or with good motion compensation.
It is also found that, in the above prior art, for any of Quantization Matrix used in different applications, the first value of quantization matrix is always set to eight, no matter whether it is low bit rate coding or high bit rate coding.
One problem with this method is the amount of information that need to be sent as part of the quantization matrix. In a typical case all 64 coefficients each of 8 bits are required. This represents a total of 512 bits. If three different Quantization Matrices are required for three bands of colour information, then the total bits will be three times of that amount. This represents too much overhead for low bit rate transmissions. It results in too long a set up time or latency in the transmissions should the matrix be changed in the middle of the transmission.
The second problem to be solved is the spatial masking of the human visual system. Noise in flat regions are more visible than noise in textured regions. Therefore applying the same matrix to all regions is not a good solution as the matrix is globally optimized but not locally adjusted to the activity of the local regions.
The third problem to be solved is the bit saving from the variable quantization matrix value for DC. The first value in Quantization matrix is decreased for higher bit rate and flat region and increased for lower bit rate and textured region.